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PREFACE 


This  Note  was  prepared  as  part  of  the  Rand  Defense  Manpower 
Research  Center,  sponsored  by  the  Office  of  the  Assistant  Secretary  of 
Defense  (Manpower,  Installations  and  Logistics) --OASD  (MIL).  With 
manpower  issues  assuming  an  ever  greater  importance  in  defense  planning 
and  budgeting,  the  Rand  study  program  seeks  to  develop  broad  strategies 
and  specific  solutions  for  dealing  with  present  and  future  defense 
manpower  problems. 

The  1979  Reserve  Force  Studies  Surveys  were  designed  and  conducted 
under  the  Rand  program  to  provide  the  Office  of  the  Deputy  Assistant 
Secretary  of  Defense  (Reserve  Affairs) --ODASD  (RA)--and  the  Army  reserve 
components  with  data  to  support  policy  formulation  and  research  on 
reserve  force  manning,  readiness,  and  unit  locations.  The  survey  was 
administered  to  the  enlisted  personnel  and  unit  commanders  in  a  sample 
of  441  Army  National  Guard  and  Army  Reserve  units. 

Three  earlier  Rand  Notes  describe  the  1979  surveys  in  detail: 

•  Zahava  D.  Doering,  David  W.  Grissmer,  Jennifer  A.  Hawes,  1979 
Reserve  Force  Studies  Surveys:  Survey  Design,  Sample  Design 
and  Administrative  Procedures ,  N-1749-MRAL,  August  1981. 

•  Zahava  D.  Doering,  David  W.  Grissmer,  Jennifer  A.  Hawes,  1979 
Reserve  Force  Studies  Surveys:  User's  Manual  and  Codebooks, 
N-1755-MRAL,  September  1981. 

•  Jennifer  A.  Hawes,  1979  Reserve  Force  Studies  Surveys: 

Descript  ion  and  Evaluation  of  Survey  Procedures ,  N-1750-MRAL, 
September  1981. 

The  present  Note,  the  last  in  the  series  documenting  the  1979  survey 
details  the  weighting  procedures  used  to  correct  for  the  bias  stemming 
from  nonresponse  to  the  survey.  It  describes  the  sample  selection  and 
explains  the  simple  weighting  scheme.  The  weighting  will  allow  analysts 
to  compute  more  accurate  population  statistics  from  the  data. 


The  work  described  in  this  final  Note  was  conducted  under  Task 
Orders  82-III-I  and  83-III-I,  Reserve  Forces  Attrition  and  Unit  Location 
Analysis,  in  Contract  MDA903-80-C-0652 ,  and  under  Contract  MDA903-83-C- 
0047.  It  should  be  useful  to  analysts  using  data  from  the  survey  to 
study  reserve  force  policy  issues  and  to  those  involved  in  the  design 
of  future  reserve  surveys. 


SUMMARY 


The  1979  Reserve  Force  Studies  Surveys  were  administered  to 
enlisted  personnel  and  unit  commanders  in  a  sample  of  4.+  1  units  during 
November  1979  and  July  1980.  Data  were  collected  through  four  survey 
questionnaires:  Form  1  solicited  information  from  junior  enlisted 

personnel  (E-l  to  E-4)  ,  Form  2  from  senior  enlisted  personnel  (E-5  to 
E-9),  and  Form  3  from  unit  commanders;  Form  4,  completed  by  a  unit 
technician  or  the  unit  commander,  sought  information  about  the  unit. 

The  surveys  collected  (1)  data  on  factors  operating  within  the  unit 
and  the  surrounding  community  which  could  explain  differences  in  reserve 
force  unit  manning  and  readiness  levels;  (2)  descriptive  statistics  from 
a  representative  sample  of  the  reserve  force  enlisted  and  unit  commander 
population;  (3)  data  on  factors  influencing  enlistment  and  reenlistment 
decisions;  and  (4)  a  baseline  data  set  for  a  reserve  force  population 
sample  that  can  be  monitored  in  the  future  to  study  reserve  force 
attrition. 

To  gather  data  with  which  to  characterize  the  Army  National  Guard 
and  Army  Reserve  populations,  we  drew  a  random  sample  of  units.  To 
identify  factors  that  distinguish  units  at  full  strength  from  units 
below  strength,  we  chose  a  second,  case  study  sample.  We  administered 
the  four  survey  questionnaires  to  each  sample. 

The  random  sample  consisted  of  113  Army  National  Guard  units  and 
109  Army  Reserve  units.  For  the  National  Guard,  we  drew  a  simple  random 
sample.  However,  because  the  Army  Reserve  had  a  disproportionate  number 
of  small  units  (fewer  than  40  reservists),  we  chose  a  stratified  random 
sample,  with  unit  size  the  stratification  variable. 

The  case  study  sample  was  selected  in  steps.  Units  were  first 
classified  by  type  (function,  and  table  of  organization  and  equipment); 
only  those  types  with  a  minimum  of  25  units  were  eligible  for  selection. 
Next,  we  chose  six  types  of  units  from  among  the  Army  Reserve  and  Army 
National  Guard  components  such  that  (a)  these  types  reflected  both 
combat  and  support  missions  and  (b)  each  type  showed  a  reasonable 
dispersion  of  strength  levels  across  individual  units.  Approximately  45 


units  of  each  type  were  chosen.  Some  40,000  enlisted  personnel  were 
sampled  with  approximately  equal  numbers  in  the  random  and  case  study 
samples . 

The  survey  response  rates  differed  widely  across  the  four  survey 
forms.  In  both  components,  well  over  80  percent  of  the  Unit  and  Unit 
Commander  surveys  were  completed,  thanks  in  part  to  the  more  intensive 
follow-up  of  these  forms.  Enlisted  personnel,  however,  returned  only  44 
to  67  percent  of  the  questionnaires,  depending  on  the  survey  and  the 
component . 

Since  nonresponse  to  some  survey  forms  was  fairly  high,  the  data 
contained  possible  bias.  Therefore,  we  had  to  weight  the  data  before  we 
used  them.  This  note  considers  weighting  only  with  regard  to  the 
randomly  sampled  units  and  enlisted  personnel.1 

The  information  on  these  four  survey  forms  can  be  used  to  describe 
the  unit  and  population  of  the  Army  reserve  component.  Because  we  have 
certain  unit  data  and  demographic  and  military  rank  personnel  variables 
across  the  entire  reserve  unit  and  personnel  population  universe,  we  can 
adopt  poststratification  weighting  procedures  to  compensate  for  missing 
surveys.  Poststratification  is  basically  an  adjustment  or  correction 
based  on  the  representativeness  of  the  survey  return  sample  with  the 
population  universe.  Weights  are  used  in  the  estimation  process  in  an 
effort  to  make  the  returned  sample  more  representative  of  the 
population. 

The  chi-square  test  was  used  to  determine  the  representativeness  of 
the  returned  unit  survey  sample  as  measured  by  several  variables 
relating  to  unit  strength  and  unit  size.  The  Army  National  Guard  sample 
closely  resembled  the  population;  the  Army  Reserve  sample,  however. 


lEach  survey  form  administered  under  a  different  sampling  plan  (4 
forms  x  2  sampling  plans  =  8  potential  weighting  schemes)  needed 
separate  consideration  with  regard  to  nonresponse  bias  and  weighting. 
The  unit  commander  forms  administered  under  the  random  and  nonrandom 
sampling  plan  could  not  be  weighted,  since  data  on  characteristics  of 
the  general  unit  commander  population  were  not  available.  However,  the 
high  response  rate  for  this  group  makes  nonresponse  bias  potentially 
less  troublesome.  The  two  enlisted  forms  and  unit  survey  administered 
under  the  case  study  sampling  plan  are  not  explicitly  considered  here. 
No  analytical  interest  exists  in  estimating  population  means  for  the 
case  study  sample  universe.  Rather,  data  from  each  unit  sampled  may 
eventually  have  to  be  weighted  so  that  units  may  bo  compared.  Such 
weighting  is  beyond  the  scope  of  the  present  note. 


differed  statistically  from  the  population  because  of  both 
stratification  and  nonresponse.  Relative  to  the  population,  the  sample 
evidently  underrepresented  small  units  and  overrepresented  large  units. 

We  used  authorized  unit  strength  as  the  weighting  variable  and 
computed  poststratification  weights  for  the.  units  (grouped  into  unit 
sizes  of  0  to  40,  41  to  100,  101  to  160,  and  over  161)  as  the  ratio  of 
expected  to  observed  frequency  counts  in  each  of  the  form  cells. 

Weights  ranged  from  2.19  for  the  smallest  units  to  0.46  for  the  largest. 
The  use  of  these  weights  eliminated  the  statistical  difference  between 
the  survey  unit  file  and  the  survey  population  file. 

Deriving  weights  was  less  straightforward  for  the  personnel  file 
than  for  the  unit  file.  Potential  weighting  candidates  included  such 
demographic  variables  as  age,  race,  sex,  pay  grade,  and  education.  The 
chi-square  test  indicated  that  bias  existed  in  response  by  age,  race, 
education,  and  pay  grade.  Indeed,  the  distribution  of  the  last  two 
variables  in  the  survey  sample  was  exceedingly  skewed  in  relation  to  the 
population  distribution. 

We  learned,  however,  that  the  updating  of  the  personnel  tapes 
lagged  with  respect  to  these  variables.  Thus,  the  survey  data  represent 
the  enlisted  population  more  accurately  than  the  data  contained  in  the 
master  personnel  file.  We  then  computed  poststratification  weights 
using  only  age  group,  sex,  and  race  (a  total  of  20  cells,  including  five 
for  age,  two  for  sex,  and  two  for  race).  These  weights  ranged  from  .81 
to  2.50. 

Weighting  the  data  insures  only  the  representativeness  of  the 
returned  sample  on  the  weighting  variables  themselves.  It  leaves  an 
unknown--but  presumably  smaller--bias  in  other  variables  for  which 
comparable  sample  and  population  data  do  not  exist. 

The  weighted  data  from  the  random  unit  file  can  be  used  to  describe 
and  analyze  several  interesting  characteristics  of  Army  Reserve  and 
National  Guard  units  not  previously  available.  These  include 
characteristics  of  the  surrounding  community,  such  as  size,  type  (inner 
city,  suburb,  etc.),  presence  of  educational  institutions,  number  and 
type  of  competing  reserve  units,  and  proximity  to  military  exchanges  and 
military  training  facilities.  In  addition,  the  unit  file  will  contain 
data  on  the  state  and  federal  military  benefits  available  to  units,  as 


well  as  unit  performance  data,  such  as  combined  training  with  active 
units  and  overseas  training. 

The  two  weighted  personnel  surveys  will  provide,  for  the  first 
time,  a  comprehensive  description  of  Army  Reservists  and  National 
Guardsmen.  Data  will  include  military  history,  civilian  labor  force 
experience,  civilian  job  characteristics,  family  statistics,  enlistment 
and  reenlistment  motivation,  employer  leave  policy,  and  several 
attitudinal  variables  describing  equipment  training  and  unit  problems. 
Beside  simple  descriptive  statistics,  the  data  will  support  the 
development  of  statistical  models  of  reenlistment  behavior  from  first 
term  to  career,  attrition  behavior  during  first  term,  and  unit  manning 
levels.  In  the  last  case,  community  characteristics  that  lead  to 
sustained  high  manning  levels  can  be  analyzed. 
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Table  3 

CASE  STUDY  SAMPLE  BY  FUNCTION,  NUMBER  OF  UNITS, 
AND  AVERAGE  STRENGTH  LEVEL 


Function 

Total 
Number  of 
Uni  ts 

Average 

Strength 

Level 

Number  of 

Sample 

Units 

• 

Army  National  Guard 

Armor 

389 

.83 

44 

Engineer 

410 

.92 

28 

• 

Infantry 

666 

1 .  10 

47 

Army  Reserve 

Combat  support 

197 

.89 

51 

Transport at  ion 

128 

.  77 

49 

Engineer 

262 

.  80 

27 

• 

NOTE:  The  average 

strength  level 

is  the  ratio 

of 

actual  strength  to  authorized  strength. 

FINAL  SAMPLE  * 

Table  5  shows  the  number  of  reservists  by  pay  grade  in  the  Army 
■.ational  Guard  and  Army  Reserve  sample  units  assumed  to  be  members  of  ' 

these  units  at  the  start  of  the  fieldwork.  Because  these  personnel  _ 

counts  were  obtained  several  months  prior  to  the  fieldwork,  they  must  be 
considered  estimates  of  the  numbers  of  individuals  in  the  survey  sample. 

Table  6  shows  the  number  of  units  of  each  component  in  each  sample. 
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Table  2 


RANDOM  SAMPLE  OK  ARMY  NATIONAL  GUARD 
AND  ARMY  RESERVE  UNITS 


Number  of  Units  Number  of  Personnel 

Unit  Size 

Universe  Sample  Uni  vers.'.  Sample 

Army  National  Guard3 

0  to  40 

205 

4 

4,700 

140 

41  to  100 

1  , 108 

46 

73,800 

3,036 

101  to  161 

1,305 

46 

141,900 

5,000 

Over  161 

515 

17 

84,200 

2,697 

Total 

3,133 

113 

304,600 

10,873 

Army  Reserve 


0  to  40 

1,576 

27 

23,800 

557 

41  to  100 

594 

26 

35,000 

1,471 

101  to  161 

393 

27 

34,800 

2,413 

Over  161 

351 

29 

85,700 

3,523 

Total 

2,914 

109 

180,300 

7,964 

Simple  random  sample  of  reserve  units. 
Sample  stratified  by  unit  size. 


Within  each  selected  functional  type,  units  were  arrayed  by  average 
strength  level  computed  as  the  ratio  of  actual  to  authorized  strength. 
They  were  then  divided  into  high,  medium,  and  low  strength,  based  on 
frequency  distributions  for  each  type  of  unit.  Individual  units  within 
each  unit  type  were  predominantly  selected  from  the  high  and  low 
strength  groups.  Table  4  lists  the  selected  units  by  function  and 
average  strength  level. 


For  the  National  Guard,  a  simple  random  sample  of  113  units  was 
selected,  or  3.6  percent  of  the  units.  Since  a  purely  random  sample  of 
units  would  have  produced  a  large  number  of  small  Army  Reserve  units,  we 
stratified  these  units  into  four  unit  size  groups  and  assigned  different 
probabilities  for  selection  to  each  category.  Within  a  category,  units 
were  chosen  randomly. 

Tlie  total  Selected  Reserve  sample  consisted  of  109  units  (3.7 
percent  of  all  Selected  Reserve  units)  almost  evenly  divided  among  the 
strata.  Table  2  shows  the  distribution  of  the  selected  random  sample 
for  the  two  components  by  unit  size. 

CASE  STUDY  SAMPLE  SELECTION 

The  type  of  units  included  in  the  case  study  samples  had  to  meet 
several  criteria.  First,  a  sufficient  number  of  units  with  identical 
functions  and  tables  of  organization  (TOE)  had  to  exist.  To  obtain  a 
reasonable  geographic  dispersion  and  allow  for  different  organizational 
environments,  a  minimum  threshold  of  25  units  was  established. 

Furthermore,  the  selected  unit  types  had  to  be  such  that  the 
average  strength  for  all  units  of  that  type  was  significantly  below  full 
strength.  Also,  individual  units  within  a  group  of  units  of  the  same 
type  had  to  show  a  reasonable  dispersion  of  strength  levels,  i.e.,  some 
units  at  or  above  full  strength  and  others  significantly  below  strength. 
Finally,  both  combat  and  support  types  of  units  had  to  be  included. 

The  six  unit  types  for  the  case  studies  were  selected  from  lists  of 
units  sorted  by  component  (National  Guard  and  Army  Reserve),  branch  or 
function  (Infantry,  Armor,  Transportation,  etc.),  authorized  strength, 
and  actual  strength.  For  the  Army  National  Guard,  Infantry,  Armor,  and 
Engineer  units  were  chosen  for  the  Army  Reserve,  Transportation, 
Engineer,  and  Combat  Support. 

As  shown  in  Table  3,  between  45  and  50  units  were  selected  from 
each  unit  type.  Since  Engineer  units  were  selected  from  both  the  Army 
Reserve  and  National  Guard,  smaller  samples  from  each  component  were 
considered  appropriate. 


Table  1 


ENLISTED  PERSONNEL  OF  THE  ARMY  NATIONAL  GUARD  AND 
ARMY  RESERVE  BY  YEARS  OF  SERVICE  AND  PAY  GRADE 
(In  thousands) 


Years  of 
Service 

E- 

Pay  Grade 

1  to  E-4  E-5  to  E-9 

Total 

Army 

National  Guard 

1  to  6 

139 

24 

163 

7  to  12 

20 

77 

97 

13  to  20 

1 

28 

29 

Over  20 

0 

18 

18 

Total 

160 

147 

307 

Army  Reserve 

1  to  6 

63 

17 

80 

7  to  12 

10 

44 

54 

13  to  20 

0 

13 

13 

Over  20 

0 

8 

8 

Total 

73 

82 

155 

Finally,  the  resources  for  printing,  distributing,  collecting, 
editing,  and  scanning  the  surveys  were  limited.  Fortunately,  we  had 
sufficient  resources  to  obtain  a  statistically  adequate  sample  in  even 
the  smallest  cells.  Resources  were  available  to  field  40,000  surveys. 

Approximately  one-half  of  the  surveys  were  devoted  to  the  random 
sample  and  one-half  to  the  case  study  sample.  This  distribution  allowed 
approximately  equal  samples  of  110  units  each  in  the  National  Guard  and 
Army  Reserve  for  the  random  sample.  Assuming  a  70  percent  response 
rate,  this  sample  size  would  provide  280  responses  in  the  smallest  cell-- 
Selected  Reservists  with  over  20  years  of  service. 


RANDOM  SAMPLE  SELECTION 

Selected  Reserve  units  vary  widely  by  size  and  function.  National 
Guard  units  tend  to  be  larger,  more  uniform  in  size,  and  higher  in 
strength  levels  than  Army  Reserve  units.  They  are  also  less  diverse  in 
terms  of  functions,  tending  to  be  predominantly  combat  units. 

The  heterogeneity  of  the  Selected  Reserve  units  poses  a  problem  for 
a  random  unit  sampling  plan.  A  simple  random  sample  of  units  would 
contain  mostly  small  units:  53  percent  of  Army  Reserve  units,  but  only 
6  percent  of  National  Guard  units,  are  authorized  for  40  or  fewer 
members.  Yet  the  Army  Reserve  also  has  several  extremely  large  medical 
units,  containing  over  160  personnel. 

The  characteristics  of  personnel  in  units  of  different  size  may 
differ  widely.  To  ensure  a  sample  representative  of  units  of  different 
size  for  the  Army  Reserve,  we  stratified  the  random  sample  by  unit  size. 
For  the  National  Guard,  a  simple  random  unit  sample  was  selected. 

To  support  research  in  a  broad  range  of  policy  concerns, 
determining  sample  sizes  required  establishing  certain  minimum 
statistical  standards  for  the  number  of  required  returned  questionnaires 
in  specific  reserve  personnel  groups.  In  particular,  we  wanted 
sufficient  sample  sizes  so  that  data  would  be  available  with  which  to 
study  the  entire  reserve  life  cycle  from  entrance  to  retirement,  as  well 
as  to  examine  reserve  subpopulations  with  and  without  prior  military 
service . 

The  overall  sample  size  was  selected  to  balance  the  statistical 
concerns  of  obtaining  an  adequate  sample  for  each  stratified  cell, 
whose  total  population  is  shown  in  Table  1,  with  certain  political  and 
budgetary  constraints.  Since  senior  personnel  are  a  smaller  proportion 
of  the  total  reserve  personnel,  the  statistical  concern  focused  on 
obtaining  an  adequate  sample  of  senior  personnel. 

The  political  constraint  involved  distributing  the  administrative 
burden  evenly  between  the  National  Guard  and  the  Army  Reserve.  This 
dictated  that  roughly  equal  numbers  of  units  be  sampled  in  each 
component . 
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II.  THE  SAMPLING  PLANS1 


To  gather  data  with  which  to  characterize  the  Army  Reserve  and  Army 
National  Guard  populations,  we  drew  a  random  sample  of  units  and 
administered  questionnaires  to  each  unit  member.  Although  statistically 
less  efficient  than  simple  individual  random  sampling,  this  method  was 
thought  to  be  less  expensive  and  likely  to  achieve  higher  response 
rates.  The  determinants  of  unit  manning  levels  could  be  modeled  on  the 
basis  of  sample  data,  and  the  models  could  be  used  to  predict  effects  of 
policies  applied  to  all  reserve  units. 

To  further  identify  factors  that  distinguish  units  at  full  strength 
from  units  below  strength,  we  chose  a  second,  case  study  sample.  Unit 
manning  levels  are  known  to  depend  on  the  size  and  type  of  the  unit. 
Smaller  units  (some  have  as  few  as  ten  members)  are  much  easier  to  man 
than  larger  units.  Also,  other  things  equal,  support  units 
(transportation,  supply,  and  maintenance,  for  example)  have  higher 
manning  levels  than  combat  units.  We  wanted  to  control  for  these 
factors  so  as  to  isolate  other  factors  important  to  manning  and 
readiness . 

We  chose  six  types  of  units  and  sampled  between  27  and  50  units  of 
each  type.  For  each  case  study  sample,  we  chose  some  units  that  were  up 
to  strength  and  some  that  were  far  below  strength.  The  case  study 
approach  provided  the  best  opportunity  for  identifying  the  invisible 
factors  that  may  affect  unit  manning  policy. 

The  combination  of  sampling  requirements  resulted  in  the  selection 
of  four  separate  and  independent  samples.  A  simple  random  sample  of 
units  was  selected  from  the  Army  National  Guard  and  a  stratified  random 
sample  from  the  Army  Reserve  to  satisfy  the  requirements  for  a  random 
sample  of  individuals  and  a  random  unit  sample.  Two  case  study  unit 
samples  were  chosen--one  each  from  the  National  Guard  and  Army  Reserve. 
Each  case  study  sample  consisted  of  three  groups  of  specific  unit  types. 


‘See  Zahava  D.  Doering,  David  W.  Grissmer,  Jennifer  A.  Hawes,  1979 
Reserve  Force  Studies  Surveys:  Survey  Design,  Sample  Design  and 
Administrative  Procedures,  N-1749-MRAL,  August  1981,  for  a  more  complete 
description  of  the  sampling  plan. 
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the  reservist’s  past  military  experience--both  active  and  reserve-- 
and  his  perceptions  of  the  unit  environment  and  manning  problems. 

Form  3,  1979  Reserve  Force  Commander  Survey,  was  completed  by  all 
unit  commanders  of  the  sampled  units.  This  survey  collected  information 
about  the  commanders'  characteristics,  military  and  civilian 
backgrounds,  and  opinions  about  unit  activities  and  environment. 

Form  4,  1979  Reserve  Force  Unit  Survey,  covered  basic  factual 
information  about  each  sampled  unit.  This  information  was  provided  by 
either  the  unit  commander  or  another  unit  member  who  was  familiar  with 
unit  data,  usually  the  unit  technician  or  unit  clerk. 

Planning  for  the  surveys  began  in  January  1979.  The  surveys  were 
fielded  nationwide  in  late  November  1979  to  the  sampled  224  Army 
National  Guard  units  and  217  Army  Reserve  units,  consisting  at  the  time 
of  the  survey  of  approximately  39,000  enlisted  personnel.  Commanders  of 
the  selected  units  were  responsible  for  collecting  data  during  regular 
monthly  drill  assemblies.  Each  unit  commander  was  responsible  for 
completing  one  Commander  Survey  and  one  Unit  Survey  and  for 
administering  Personnel  Surveys  to  all  enlisted  unit  members. 

The  surveys  were  scheduled  for  completion  in  December  1979.  The 
logistics  of  survey  administration  during  drill  assemblies  and  such 
operational  requirements  as  unit  maneuvers  and  emergency  call-ups 
delayed  completion  of  the  data  collection  until  July  1980. 3  After  the 
survey  was  completed,  the  questionnaires  were  edited,  processed,  and 
converted  into  machine-readable  files. 

3See  Hawes,  September  1931,  for  details  of  survey  administration 
and  survey  response. 
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Little  information  is  currently  available  concerning  important 
aspects  of  the  civilian  life  of  reservists,  including  factors  that 
influence  enlistment  and  reenlistment  decisions.  These  factors  include 
civilian  job  characteristics,  family  income,  past  military  history,  and 
potential  conflicts  between  civilian  job  requirements  and  reserve 
participation. 

The  second  half  of  the  sample  was  chosen  to  study  unit  manning, 
readiness,  and  location.  We  collected  data  describing  reserve  unit 
manning  and  readiness  characteristics,  including  recruiting  and  training 
resources  and  strategies,  equipment  and  facilities  available  for 
training,  and  unit  commander  attitudes  and  assessments.  Community  data 
include  local  population,  types  and  wages  of  available  jobs,  size  and 
characteristics  of  nearby  educational  institutions,  and  the  existence  of 
other  reserve  force  units. 

Because  of  the  multiple  purposes  of  these  surveys  and  the  large 
number  of  questions  required  to  address  some  of  the  topics,  we  devised 
four  separate  questionnaires  for  the  surveys.2  These  were  designated 
forms  1  to  4. 

Form  1,  1979  Reserve  Force  Personnel  Survey  for  Grades  E~1  to  E-4, 
collected  data  from  all  junior  enlisted  personnel  who  were  members  of 
the  sample  units  at  the  time  of  the  survey  administration.  The  major 
part  of  the  information  collected  in  this  questionnaire  related  to  the 
first-term  enlistment  decision  and  to  the  characteristics  and  experience 
of  the  reservist  prior  to  enlistment.  A  group  of  possible  indicators  of 
attrition  and  reenlistment  behavior  was  also  included,  along  with 
detailed  economic  and  civilian  labor  force  data. 

Form  2,  1979  Reserve  Force  Personnel  Survey  for  Grades  E-5  to  E-9 , 
was  administered  to  all  senior  enlisted  personnel,  including  unit 
technicians  and  training  noncommissioned  officers  (NCOs),  who  were 
members  of  the  sampled  units  at  the  time  of  the  survey  administration. 
This  questionnaire,  like  Form  1,  also  collected  detailed  economic  and 
civilian  labor  force  data  and  possible  indicators  of  reenlistment 
behavior.  In  addition,  a  major  portion  of  the  questionnaire  focused  on 

2The  survey  forms  are  reproduced  in  Doering  et  al.,  August  1981, 
pp.  72-169. 


to  be  used  with  the  random  sample  files  (both  the  unit  and  personnel 
files)  and  justifies  both  their  derivation  and  use.  Section  IV  suggests 
the  kind  of  analysis  possible  with  these  data. 

The  1979  Reserve  Force  Studies  Surveys  collected  the  following 
kinds  of  data: 

•  Data  on  factors  operating  within  the  unit--factors  that,  when 
combined  with  data  describing  the  surrounding  community, 
explain  differences  in  reserve  force  unit  manning  and  readiness 
levels . 

•  Data  on  factors  influencing  enlistment  and  reenlistment 
decisions . 

•  Descriptive  statistics  from  a  representative  sample  of  the 
reserve  force  enlisted  and  unit  commander  population. 

•  A  baseline  data  set  for  a  reserve  force  population  sample  that 
can  be  monitored  in  the  future  to  analyze  force  attrition. 

The  collected  data  provide  a  unique  description  of  the  men  and 
women  currently  in  the  Army  reserve  components.  The  factors  underlying 
the  success  or  failure  of  unit  manning  and  readiness  can  be  analyzed  on 
the  basis  of  the  information  gathered  in  the  surveys. 

Our  research  strategy  was  to  sample  the  personnel  and  commanders  of 
44-1  Army  National  Guard  and  Army  Reserve  units  of  different  authorized 
strength  levels,  in  different  geographical  areas,  and  in  different  kinds 
of  communities.  Approximately  one-half  of  the  units  were  chosen 
randomly  so  that  average  unit  and  personnel  characteristics  could  be 
studied.  The  rational  for  the  sample  plans  and  the  survey  response  rate 
are  described  in  detail  in  earlier  Rand  Notes.1  In  this  Note,  we 
summarize  the  sampling  plans  in  Section  II. 

‘See  Zahava  D.  Doering,  David  W.  Grissmer,  Jennifer  A.  Hawes,  1979 
Reserve  Force  Studies  Surveys:  Survey  Design,  Sample  Design  and 
Administrative  Procedures,  N-1749-MRAL,  August  1981.  See  also  Zahava  D. 
Doering,  David  W.  Grissmer,  Jennifer  A.  Hawes,  1979  Reserve  Force 
Studies  Surveys:  User's  Manual  and  Codebooks ,  N-1755-MRAL,  September 
1981;  and  Jennifer  A.  Hawes,  1979  Reserve  Force  Studies  Surveys: 

Descript  ion  and  Evaluation  of  Survey  Procedures ,  N-1750-MRAL,  September 
1981. 


Weighting  procedures  must  be  documented  so  that  users  of  the  survey 
data  can  judge  the  applicability  of  the  data  to  specific  policy  issues. 
We  can  weight  our  data  because  we  chose  military  survey  samples  from 
personnel  files  containing  data  on  the  demographic  and  military 
characteristics  of  both  the  population  universe  and  the  specific  survey 
sample.  By  combining  these  data  with  data  from  the  returned  surveys,  we 
are  able  to  weight  and  improve  estimates  from  the  survey  data. 

This  note  reports  the  magnitude  of  nonrepresentativeness  between 
the  returned  survey  population  and  the  reserve  population  for  several 
demographic  status  variables.  Determining  response  rate  patterns  under 
different  administration  modes  becomes  important  in  designing  future 
reserve  surveys. 

Survey  response  on  personnel  surveys  proved  lower  among  reservists 
who  are  young  and  white,  but  was  nearly  identical  for  males  and  females. 
This  pattern  was  similar  for  both  National  Guard  and  Army  Reserve 
personnel.  One  explanation  for  the  pattern  of  nonresponse  is  the  more 
frequent  absence  of  younger  reservists  from  drills,  owing  either  to 
initial  active  duty  training  or  early  attrition. 

Response  rates  could  not  be  compared  for  groups  with  different 
education  levels  or  military  ranks,  since  data  on  the  personnel  files 
are  not  current.  Response  patterns  on  the  unit  characteristics  survey 
showed  Army  National  Guard  units  that  returned  surveys  to  be 
representative  of  all  Guard  units;  in  contrast,  large  Army  Reserve  units 
responded  better  than  smaller  units. 

We  developed  simple  weights  for  the  unit  survey  based  on  unit  size 
for  the  Army  Reserve  units.  Personnel  survey  weights  were  developed 
separately  for  National  Guard  and  Army  Selected  Reserve  personnel,  based 
on  race,  age,  and  sex.  These  weights  ensure  that  the  survey  data  more 
accurately  describe  the  characteristics  of  reserve  units  and  reserve 
personnel . 

The  remainder  of  this  section  summarizes  the  objectives, 
administration,  and  content  of  the  surveys.  Section  II  describes  the 
sampling  plan  and  sample  selection  process.  Section  III  discusses  the 
bias  that  nonresponse  introduced  in  some  important  variables  and 
explains  the  need  for  corrective  weights.  It  also  describes  the  weights 
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I.  INTRODUCTION 


The  1979  Reserve  Force  Studies  Surveys  were  undertaken  to  help  fill 
critical  gaps  in  information  about  reserve  force  personnel  and  units. 
This  information  deficit  was  substantial,  partly  because  reserve 
personnel  had  not  been  surveyed  systematically.  Although  active 
personnel  have  been  surveyed  periodically  since  the  early  1970s  through 
the  Department  of  Defense  Survey  of  Active  Personnel,  the  Armed  Forces 
Entrance  and  Examining  Station  survey,  and  numerous  service-specific 
surveys,  the  Department  of  Defense  neither  includes  reserve  personnel  in 
these  surveys  nor  surveys  them  separately  on  a  continuing  basis. 

This  lack  of  reserve  force  surveys  is  unfortunate  because  survey 
data  are  more  critical  to  reserve  manpower  policy  analysis  than  to 
active  force  analysis.  For  active  force  personnel,  an  extremely  rich 
and  detailed  set  of  historical  data  are  collected  and  maintained  in 
computerized  personnel  files.  These  data  can  support  many  kinds  of 
analysis  without  the  need  for  supplementary  survey  data. 

Reserve  personnel  files  not  only  contain  fewer  data  than  active 
force  files,  but  the  data  that  are  usually  collected  are  less  valuable 
in  analyzing  reserve  personnel  policy  decisions.  The  reserve  data  are 
less  valuable  because  reserve  personnel  behavior  depends  to  a  great 
extent  on  the  course  of  their  civilian  jobs  and  family  life- -areas  in 
which  only  surveys  can  provide  the  necessary  data. 

At  the  same  time,  surveying  reserve  personnel  presents  certain 
challenges  stemming  from  their  limited  availability  and  weaker 
attachment  to  the  military  job.  Obtaining  the  adequate  response  rates 
needed  for  analytic  credibility  can  be  difficult. 

Since  moderate  levels  of  nonresponse  characterize  reserve  surveys, 
the  data  must  be  weighted.  Without  adjustment,  obvious  differences  in 
representation  of  different  groups  in  the  population  and  survey  sample 
may  threaten  credibility.  Although  postsurvey  weighting  can  correct 
this  representation  for  certain  variables,  it  nevertheless  leaves  an 
unknown,  but  presumably  smaller,  bias  in  the  remaining  variables. 
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Table  4 

CASE  STUDY  SAMPLE  BY  FUNCTION  AND  STRENGTH  LEVEL 


Low 

Strength 
(under  . 70) 

Medium 
Strength 
(.70  to  .90) 

High 

Strength 
(over  .90) 

Total 

Army  National  Guard 

Armor 

18 

8 

18 

44 

Engineer 

12 

4 

11 

27 

Infantry 

22 

6 

19 

47 

Army  Reserve 

Combat  support 

24 

16 

10 

^0 

Transportation  Corps 

17 

13 

15 

45 

Engineer 

11 

9 

6 

26 

NOTE:  The  average  strength  level  is  the  ratio  of  actual  strength  to 

authorized  strength.  For  the  engineer  units,  we  used  the  following 
strength  levels:  under  .50,  .50  to  .70,  and  over  .70. 


Table  5 


ESTIMATED  NUMBER  OF 
RESERVE  FORCE 

PERSONNEL  IN  THE 
SURVEY  SAMPLES 

1979 

Category 

National 

Guard 

Army 

Reserve 

Unit  commanders 

224 

217 

Enlisted  personnel 

E-l  to  E-4 

11,275 

10,193 

E-5  to  E-9 

8,795 

8,756 

Total 

20,072 

18,949 

13 


Table  6 

FINAL  SAMPLE  STRATIFICATION  OF  ARMY  NATIONAL  GUARD 
AND  ARMY  RESERVE  UNITS 


Number  of 
Sample  Units 


Army  National  Guard 
Case  study  sample 

Armor  .  42 

Engineer  .  25 

Infantry  .  44 

Total  case  study  sample  .  Ill 

Random  sample  .  113 

Total  National  Guard  sample  .  224 

Army  Reserve 

Case  study  sample 

Combat  support  .  45 

Transportation  .  45 

Engineer  .  21 

Total  case  study  sample  .  Ill 

Random  sample  .  106 

Total  Selected  Reserve  sample  .  217 

Total  sample  .  441 


III.  WEIGHTING  TO  CORRECT  FOR  NONRESPONSE  BIAS 


SURVEY  RESPONSE  RATES 

The  response  rates  for  this  study  were  computed  as  the  number  of 
questionnaires  completed  by  eligible  sample  members  divided  by  the  total 
number  of  eligible  sample  members.  These  calculations  followed  the 
formula  suggested  in  the  American  Statistical  Association  report  on 
methods  for  assessing  survey  practices.1 

Using  this  approach,  we  calculated  the  response  rate  for  the 
surveys  of  enlisted  personnel  as  the  ratio  of  the  number  of  completed 
Enlisted  Personnel  Surveys  to  the  number  of  enlisted  personnel  assigned 
to  the  sampled  units  at  the  time  of  the  survey  administration.2  For  the 
Unit  and  Commander  Surveys,  response  rates  were  calculated  as  the  number 
of  sampled  units  for  which  completed  questionnaires  were  received 
divided  by  the  total  number  of  sampled  units. 

Overall  survey  response  rates  for  each  of  the  four  questionnaires 
used  in  this  study  are  presented  in  Table  7.  The  unit  commanders  were 
I  diligent  in  completing  the  two  questionnaires  designed  for  their  use, 

namely,  the  Unit  and  Commander  Surveys.  In  both  the  Army  Reserve  and 
the  Army  National  Guard,  the  completion  rates  for  these  two  surveys  were 
over  80  percent,  well  over  our  projected  response  rate  of  70  percent  for 
I  all  surveys. 


1Barbara  A.  Bailar  and  C.  Michael  Lanphier,  Development  of  Survey 
Methods  to  Assess  Survey  Practices,  American  Statistical  Association, 
Washington,  D.C. ,  1978. 

2The  data  pertaining  to  the  number  of  enlisted  personnel  who  were 
members  of  the  sampled  units  at  the  time  of  the  survey  were  tabulated 
from  the  completed  Unit  surveys.  Using  unit  records  of  actual  personnel 
strength,  the  unit  commander  recorded  the  required  data  in  the  Unit 
Survey.  For  units  that  did  not  return  completed  Unit  Surveys 
(approximately  18  percent  of  the  sample),  the  personnel  data  were 
abstracted  from  the  reserve  personnel  files  available  at  the  time  of 
sample  selection,  i.e.,  as  of  July  or  August  1979,  based  on  the  Reserve 
Components  Personnel  Data  System. 


i 
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Table  7 

RESPONSE  RATES  FOR  SURVEY  FORMS 

Army  National 


Survey  Form 

Guard 

Army  Reserve 

Total 

Sample 

Size 

Average 

Response 

Rate 

Sample 

Size 

Response 

Rate 

Sample 

Size 

Response 

Rate 

Unit3 

224 

80 . 8% 

217 

83.4% 

441 

82 . 1% 

Commander3 

224 

79.9% 

217 

82.5% 

441 

81.2% 

E-5  to  E-9b 

8,722 

66 . 9% 

9,099 

63 . 2% 

17,821 

65 . 1% 

E-l  to  E-4b 

11,341 

52.8% 

10,445 

44 . 2% 

21,786 

48.5% 

Sample  size  is  the  number  of  units  included  in  the  survey. 

bFor  units  that  returned  completed  Unit  Surveys,  the  sample 
size  is  the  number  of  enlisted  personnel  in  each  pay-grade  group 
reported  in  the  survey.  For  units  that  did  not  return  completed  Unit 
Surveys,  sample  size  is  based  on  the  number  of  enlisted  personnel  in 
each  pay-grade  group  at  the  time  of  sample  selection,  i.e.,  as  of  July 
or  August  1979,  based  on  the  Reserve  Components  Personnel  Data  System. 


The  rate  of  returned  questionnaires  from  enlisted  unit  members, 
however,  fell  below  the  projected  70  percent  to  65  percent  for  senior 
enlisted  personnel  and  under  49  percent  for  junior  enlisted  personnel. 

The  completion  rate  of  53  to  67  percent  for  Army  National  Guard  enlisted 
personnel  was  slightly  higher  than  the  44  to  63  percent  obtained  from 
Army  Reserve  enlisted  personnel.3 

An  alternative  way  to  look  at  response  rates  is  to  examine  the  unit- 
specific  response  rates  for  the  Enlisted  Personnel  Surveys.  Our 
analysis  of  fieldwork  data  shows  that  response  rates  among  enlisted 
personnel  varied  greatly  among  sampled  units.  A  high  unit  response  rate 

’Unfortunately,  the  response  rate  to  the  Enlisted  Personnel  Surveys 
was  depressed  by  about  4  percent  as  a  result  of  the  loss  in  the  U.S. 
mail  of  five  boxes  containing  approximately  3500  completed 
questionnaires  that  were  being  sent  from  DMDC  in  Alexandria,  Virginia, 
to  the  optical  scan  contractor,  Intran,  in  Minneapolis,  Minnesota. 


-  *•  -  ---I 


indicates  that  the  unit  commander  administered  the  personnel  surveys  to 
a  large  percentage  of  the  enlisted  unit  members,  with  few  refusals  and 
absences.  Lower-than-expected  unit  response  rates  may  have  resulted 
from  individual  refusals  to  participate  or  low  attendance  at  monthly 
reserve  unit  drills. 

The  existence  of  "split"  reserve  units  may  also  have  negatively 
affected  response  rates.  We  found  that  roughly  20  percent  of  the  units 
selected  for  sampling  were  organized  into  two  or  more  sections  that 
drilled  at  different  times  and  in  different  geographical  locations. 
Enlisted  response  rates  in  these  split  units  may  have  been  lower  because 
the  units  failed  to  administer  the  surveys  to  enlisted  members  at  all 
unit  drill  locations.  The  relatively  low  response  rates  for  junior 
enlisted  members  depressed  the  overall  response  rates  for  unit  enlisted 
personnel . 

NONRESPONSE  BIAS  IN  THEORY 

This  section  examines  nonresponse  bias  and  the  assumptions 
underlying  various  compensation  procedures . “  Consider  a  simple  random 
sample  of  size  n  drawn  from  a  population  of  size  N.  We  are  interested 
in  estimating  the  population  mean  for  a  single  variable  Y. 

Let  R  be  the  number  of  respondents  and  M  the  number  of 

nonrespondents  in  the  population,  so  that  R  +  M  =  N;  the  corresponding 

sample  figures  are  r  and  m  with  r  +  m  =  n.  Let  R  =  R/N  and  R  =  M/N  be 

the  proportions  of  respondents  and  nonrespondents  in  the  population; 

then  r  =  r/n  and  m  =  m/n  are  the  corresponding  sample  response  and 

nonresponse  rates.  The  population  mean  for  the  variable  Y  is  Y  =  RY  + 
_____  ** 
MY^,  i.e. ,  the  population  mean  is  a  weighted  sum  of  the  means  for 

respondents  and  nonrespondents,  Y^  and  Y^,  respectively,  the  weights 

being  the  proportions  of  respondents  and  nonrespondents. 

If  no  compensation  is  made  for  nonresponse,  the  respondent  sample 
mean,  y  ,  is  used  to  estimate  Y.  Its  bias  is  given  by: 


“This  section  draws  heavily  on  Graham  Kalton,  "Compensating  for 
Missing  Survey  Data,"  Survey  Research  Center,  Institute  for  Social 
Research,  The  University  of  Michigan,  Research  Report  Series,  19S3. 
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Bias  (yr)  =  YR  -  Y  =  M(YR  -  Y^) 


This  shows  that  y^  is  approximately  unbiased  if  either  the  proportion  of 
nonrespondents  Fi  is  small  or  Y^  is  close  to  YR,  i.e.,  the  means  for  both 
the  respondent  and  nonrespondent  groups  are  similar.  If,  as  in  the 
Enlisted  Personnel  Surveys,  the  proportion  of  nonrespondents  is  large, 
then  the  suitability  of  y^  as  an  estimator  of  Y  depends  on  the 

assumption  that  Y  and  Y  are  close  in  value.  In  other  words,  the 

r  r  m 

choice  not  to  compensate  for  missing  data  involves  the  implicit 
assumption  that  respondents  and  nonrespondents  have  similar  means. 

As  Kalton  points  out,  given  fairly  sizable  nonresponse  rates,  it  is 
far  better  to  adopt  some  form  of  compensation  procedures  based  on 
explicit  and  realistic  assumptions  rather  than  simply  to  analyze  the 
respondent  data. 

Different  types  of  weighting  adjustments  may  be  used  to  ensure  that 
the  survey  data  conform  closely  to  the  survey  population.  The  method 
used  here  is  poststratification  weighting.5  The  development  of 
poststratification  weights  requires  that  information  be  available  on  the 
distribution  of  the  survey  population  over  different  subgroups  ot  the 
sample,  often  referred  to  as  weighting  classes  The  weighting  factors 
are  then  based  on  the  population  totals  for  the  weighting  classes.  The 
data  from  the  respondent  sample  are  then  weighted  so  that  their  weighted 
distribution  over  the  classes  conform  to  that  of  the  original  survey 
populat ion . 

Poststratification  or  stratification  after  selection  is  basically 
an  adjustment  or  correction  of  the  mean.  The  effect  of  stratification 
is  introduced  into  the  estimation  process  in  an  effort  to  make  the 
sample  proportionate  to  the  population.  This  is  an  example  of  improving 
the  estimator  by  the  proper  utilization  of  ancillary  sources  of 
in  format  ion . 

5For  a  discussion  of  this  method,  see  Leslie  Kish,  Survey  Sampling, 
John  Wiley  and  Sons,  Inc.,  New  York,  New  York,  1965. 
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For  example,  assume  that  we  can  classify  sample  cases  into  H  strata 
and  also  obtain  the  proportions  and  of  these  strata  in  the  sample 
and  population,  respectively.  The  weighted  population  mean  Y1  is  given 
by: 


Y'  =  I  W,  Y,  =  W,Y,  +  W„Y„  + 
h  n  11  Z  Z 


W  Y 
HI! 


i.e.,  the  population  mean  is  equal  to  the  sum  of  the 
multiplied  by  their  weights  W^,  where  Z  =  1.  The 
mean , 


H  strata  means  Y, 
n 

uncorrected  sample 


y 


wh  V 


has 


Bias 


(Y)  =  (I  whEyh 


-  z 


MhV 


=  Z  Wh(Eyh  -  Yh)  +  z  (wh  -  Wh)Eyh 
=  Z  Wh(Eyh  -  Yh)  +  Z  (wh  -  Wh)(Eyh  -  Ey) . 


where  Ey^  is  the  expected  value  of  y  among  respondents  in  stratum  h. 
The  poststratification  procedure  computes  a  weighted  sampie  mean  as: 


which  has 


Bias  (y ’ )  =  I  Wh(Eyh  -  Yh). 


This  procedure,  of  course,  cannot  guarantee  removal  of  nonresponse 
bias  from  the  sample.  If  the  nonrespondents  within  each  group  h  are  a 
random  sample  of  the  total  respondent  and  nonrespondent  sample  for  that 
group  (choice-based  sample),  then  we  would  have  Ey^  =  Y^  and  the 
weighting  procedure  would  remove  nonresponse  bias.  However,  this 
procedure  cannot  completely  remove  bias  for  variables  for  which  data  are 
available  in  the  sample  only. 

Typically,  certain  variables  (demographic)  will  be  available  for 
both  the  population  and  sample,  and  other  variables  will  be  gathered  for 
the  sample  only.  Some  subset  of  the  former  will  be  used  to  define  the  H 
classes  and  weights.  These  weights  will  then  be  used  to  derive  means  of 
variables  found  in  the  sample  only.  No  weighting  procedure  can 
guarantee  absence  of  bias  in  the  means  of  the  latter  set  of  variables 
unless  selection  is  truly  random  within  each  class.  Since  the  causes  of 
nonresponse  are  usually  not  simple  random  procedures,  an  unknown  bias 
will  remain. 
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NONRESPONSE  BIAS  IN  THE  SURVEY 

The  random  sample  file  was  separated  into  (a)  the  unit  file 
containing  information  from  the  Form  4  questionnaire  filled  out  by  the 
unit  technician  or  unit  commander  and  (b)  the  personnel  file  containing 
the  data  from  the  Form  1  and  Form  2  questionnaires  filled  out  by  E-l  to 
E-4  and  E-5  to  E-9  reservists,  respectively.  The  Unit  Commander  survey 
for  the  random  sample  cannot  be  weighted  because  of  the  lack  of 
personnel  data  defining  the  universe  for  unit  commanders.  However,  the 
high  response  rate  for  this  questionnaire  makes  nonresponse  a  less 
troublesome  issue.  The  Unit  Commander  survey  response  rates  exceeded  80 
percent  for  each  guard  and  reserve  sample. 

Nonresponse  Bias  in  the  Random  Sample  Unit  Survey  File 

We  compared  the  Army  Reserve  and  Army  National  Guard  respondent 
unit  files  with  the  unit  survey  population  file  across  several  variables 
relating  to  unit  strength  and  unit  size.  Tables  8  and  9  present  the 
distribution  of  the  sample  and  the  population  by  enlisted  structure 
strength,  enlisted  authorized  strength,  enlisted  operating  strength,  and 
enlisted  strength  present  for  annual  training  for  the  two  components, 
respectively. 

The  distributions  appear  to  be  fairly  similar  for  the  National 
Guard  population  and  sample  units.  The  Army  Reserve  sample,  however, 
appears  to  differ  markedly  from  the  expected  stratified  sample,  with  a 
larger  response  from  larger  units. 

To  test  the  significance  of  the  observed  differences  in  the 
distribution  of  the  variables  across  the  survey  sample  and  survey 
population,  we  used  the  chi-square  (x2)  test.  The  null  hypothesis  was 
one  of  no  difference  between  the  survey  sample  and  population 
distribution  across  any  of  the  selected  variables.® 

60ne  computes  a  set  of  expected  or  theoretical  frequencies  for  the 
sample  based  on  the  assumption  that  the  null  hypothesis  is  true.  The 
observed  frequencies  are  then  compared  to  the  expected  frequencies.  The 
chi  square  is  computed  as  X2  =  I  (f0  *  ft)2/ft>  where  f0  is  the  observed 

frequency  and  f  is  the  theoretical  frequency.  If  the  computed  chi 

square  is  greater  than  the  value  of  the  chi  square  given  in  the  chi- 
square  distribution  table  for  the  appropriate  degrees  of  freedom  and  the 
selected  level  of  significance,  the  null  hypothesis  is  rejected. 


Table  8 


COMPARISON  OF  SURVEY  POPULATION  AND  FORM  4  SAMPLE 
OF  ARMY  RESERVE  UNITS 


Unit  Size 


Variable 

0-40 

41-100 

101-160 

161+ 

Total 

Enlisted  structured  strength 
Survey  population 

Survey  sample 

54.1 

25.0 

18.1 

22.7 

12.9 

21.6 

15.0 

30.7 

100.0 

100.0 

Enlisted  authorized  strength 
Survey  population 

Survey  sample 

54.1 

25.0 

20.4 

23.9 

13.4 

23.9 

12.1 

27.3 

100.0 

100.0 

Enlisted  operating  strength 
Survey  population 

Survey  sample 

57.8 

31.8 

28.3 

42.1 

10.1 

22.7 

3.8 

3.4 

100.0 

100.0 

Enlisted  present  for  annual 
training 

Survey  population 

Survey  sample 

67.1 

47.7 

26.6 

39.8 

5.2 

12.5 

1.1 

100.0 

100.0 

Table  10  presents  the  results  of  the  chi-square  test  comparing  the 
sample  and  population  distribution;  it  also  reports  the  critical  value 
of  chi  square  for  0.05  level  of  significance.  We  cannot,  in  any 
instance,  reject  the  null  hypothesis  for  the  National  Guard  units. 
However,  we  reject  the  null  hypothesis  of  no  difference  between  the 
survey  sample  and  survey  population  for  the  Army  Reserve  units. 


Table  9 

COMPARISON  OF  THE  SURVEY  POPULATION  AND  FORM  4  SAMPLE 
OF  NATIONAL  GUARD  UNITS 


Unit  Size 


Variable 

0-40 

41-100 

101-160 

161+ 

Total 

Enlisted 

structured  strength 

Survey 

population 

7.4 

28. 1 

3b.  1 

28.5 

100.0 

Survey 

sample 

2.2 

33.3 

37.6 

26.9 

100.0 

Enlisted 

authorized  strength 

Survey 

populat ion 

6.5 

35.4 

41.1 

17.0 

100.0 

Survey 

sample 

2.2 

39.8 

43.0 

15.0 

100.0 

Enlisted 

operating  strength 

Survey 

populat ion 

8.1 

50.5 

33.5 

7.9 

100.0 

Survey 

sample 

4.3 

53.8 

34.4 

7.5 

100.0 

Enlisted 

present  for  annual 

training 

Survey  population 

16.8 

63.1 

18.1 

2.0 

100.0 

Survey 

sample 

16.1 

63.4 

20.4 

-- 

100.0 

Table  10 

RESULTS  OF  CHI-SQUARE  TESTS  COMPARING  SURVEY  POPULATION 

AND  FORM  4  SAMPLE 


Army  National  Army  ) 

Guard  Reserve  0. 


Enlisted  structured  strength 
Enlisted  authorized  strength 
Enlisted  operating  strength 
Enlisted  present  for  annual 
training 


4.60 

35.98 

7.8 

3.50 

39.04 

7.8 

2.20 

29.60 

7.8 

2.30 

19.40 

7.8 

NOTE:  Each  chi-square  statistic  has  3  degrees  of  freedom. 
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We  decided  to  weight  the  Army  Reserve  unit  file  using  enlisted 
authorized  strength.  We  did  not  weight  the  National  Guard  unit  file,  as 
it  was  already  representative  of  the  population.  Poststratification 
weights  for  the  Army  Reserve  units  are  given  in  Table  11. 

Nonresponse  Bias  in  the  Random  Sample  Form  1  and  Form  2 
Combined  Personnel  File 

We  examined  the  extent  of  nonresponse  bias  in  the  personnel  sample 
composed  of  Form  1  and  Form  2  respondents  by  comparing  the  distribution 
of  the  survey  population  with  the  sample  across  various  demographic 
variables,  as  shown  in  Table  12.  Notice  that  the  survey  sample, 
irrespective  of  component,  has  fewer  young  people,  fewer  reservists  in 
the  E-l  and  E-2  pay  grades  and  a  much  higher  population  of  personnel 
with  some  college  education  than  the  survey  population.  We  computed  chi 
square  to  test  the  significance  of  these  differences;  the  results  are 
reported  in  Table  13.  As  the  table  makes  clear,  in  almost  every  case, 
we  were  forced  to  reject  the  null  hypothesis  of  no  significant 


Table  11 

POSTSTRATIFICATION  WEIGHTS  FOR  ARMY  RESERVE  UNITS 
IN  RANDOM  SAMPLE 


Enlisted 

Authorized 

Strength 

Proportion 
in  Survey 

Population 

Expected 

Frequency3 

Observed 

Frequency 

Weight^ 

0  to  40 

.54 

48 

22 

2.19 

41  to  100 

.  20 

18 

21 

0.88 

101  to  160 

.  13 

11 

21 

0.52 

Over  161 

.  12 

11 

24 

0.46 

3 

Expected  frequency  is  calculated  by  multiplying  the  total 
number  of  survey  sample  units  (88)  by  the  proportion  that  each 
cell  accounted  for  in  the  survey  population  (column  1). 


Weight  is  the  expected  frequency  divided  by  the  observed 
frequency. 
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Table  12 


COMPARISON  OF  SURVEY  POPULATION  AND  FORMS  1  AND  2 
COMBINED  ENLISTED  PERSONNEL  FILE 
BY  DEMOGRAPHIC  VARIABLE 


Army  National 

Guard 

Army  Reserve 

Variable 

Survey 

Population 

Survey- 

Sample 

Survey  Survey 

Population  Sample 

Sex 

Male 

95.4 

95.5 

86.4 

86.7 

Female 

4 . 6 

4.5 

13.6 

13.3 

Total 

100.0 

100.0 

100.0 

100.0 

Race 

B 1  ack 

18.2 

15.8 

28.4 

23.9 

Nonblack 

81.8 

84.2 

71.6 

76.1 

Total 

100.0 

100.0 

100.0 

100.0 

Age  group 

17  to  20  years 

16.5 

14.0 

12.5 

8.0 

21  to  24  years 

22. 1 

20.2 

18.3 

15.9 

25  to  29  years 

18.4 

18.6 

21.0 

23.0 

30  to  36  years 

22.6 

25.2 

27.7 

31.1 

Over  37  years 

20.4 

22.0 

20.5 

22.0 

Total 

100.0 

100.0 

100.0 

100.0 

Pay  grade 

E-l 

10.7 

7  .  1 

7.0 

4.1 

E-2 

9.9 

7  .  1 

10.9 

4 . 6 

E-3 

10.4 

10.8 

8.1 

8.2 

E-4 

21.0 

22.1 

22.0 

25.1 

E-5 

24.2 

27.5 

22.9 

27.4 

E-6 

14.7 

16.0 

15.7 

17.2 

E-7 

6.8 

7.1 

9.7 

9.3 

E-8 

1.9 

2.0 

2.9 

3.2 

E-9 

.4 

.  3 

.8 

.9 

Total 

100.0 

100.0 

100.0 

100.0 

Education 

Less  than  high  school 

32.2 

25.6 

27.1 

16.9 

High  school  graduate 

63.1 

51.8 

61.0 

49.3 

Some  college 

4.7 

22.6 

11.9 

33.8 

Total 

100.0 

100.0 

100.0 

100.0 
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Table  13 

RESULTS  OF  Cltl -SQUARE  TESTS  COMPARING  SURVEY 
POPULATION  AND  FORMS  1  AND  2  COMBINED 
ENLISTED  PERSONNEL  FILE 


Variable 

Army  National 
Guard 

Army 

Reserve 

X2 

0.05 

Race 

24.2 

41.0 

3.8 

Sex 

0.3 

0.3 

3.8 

Age 

48.9 

96.4 

9.5 

Pay  grade 

165.3 

276.5 

15.5 

Education 

4206.5 

1904.7 

6.0 

NOTE:  Chi  square  may  be  biased  because  of 

clustering. 


differences  between  the  survey  sample  and  the  survey  population.  This 
implied  that  our  respondent  sample  was  not  representative  of  the  survey 
population,  at  least  with  regard  to  these  particular  variables. 

Weighting  the  Survey  Form  1  and  Form  2  Combined  Personnel  File 

At  first  glance,  it  appeared  that  we  should  weight  by  pay  grade 
and/or  education.  We  learned,  however,  that  the  updating  of  the 
personnel  file  lagged  with  respect  to  both  of  these  variables. 

The  information  that  we  obtained  from  the  survey  might  plausibly 
have  been  more  current  than  the  distribution  shown  in  the  reserve 
personnel  files  from  which  the  survey  population  data  were  obtained.  In 
fact,  education  variables  on  this  file  often  reflect  education  at 
enlistment;  these  cases  would  account  for  the  high  chi  square.  Pay 
grade  is  likely  to  lag  by  only  a  tew  months.  We,  therefore,  stratified 
the  population  by  age  group,  sex,  and  race. 

Table  14  presents  the  comparison  of  the  sample  and  the  population 


I 


for  these  strata.  Poststratification  weights  were  then  computed,  as 
shown  in  Table  15. 
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Table  14 

COMPARISON  OF  SURVEY  POPULATION  AND  FORMS  1  AND  2 
COMBINED  ENLISTED  PERSONNEL  FILE 
BY  STRATIFICATION  VARIABLE 


Army  National 

Guard 

Army  Reserve 

Variable 

Survey 

Population 

Survey 

Sample 

Survey  Survey 

Population  Sample 

17  to  20  years  old 

16.5 

14.7 

12.6 

7.9 

Black  males 

2.8 

2.5 

2.9 

1.5 

Nonblack  males 

12.8 

11.3 

7.9 

5.5 

Black  females 

.2 

.2 

.6 

.4 

Nonblack  females 

.  7 

.  7 

1.2 

.5 

21  to  24  years  old 

22.1 

20.5 

18.3 

16.2 

Black  males 

5.2 

4.1 

5.3 

3.3 

Nonblack  males 

15.4 

14.8 

9.6 

9.1 

Black  females 

.5 

.3 

1.2 

1.5 

Nonblack  females 

1.0 

1.2 

2.2 

2.3 

25  to  29  years  old 

18.4 

18.4 

21.0 

23.1 

Black  males 

3.7 

3.1 

5.6 

5.0 

Nonblack  males 

13.5 

14.2 

11.2 

13.8 

Black  females 

.4 

.3 

1.7 

1.9 

Nonblack  females 

.8 

.8 

2.5 

2.4 

30  to  36  years  old 

22.6 

24.7 

27.7 

30.6 

Black  males 

3.4 

3.1 

6.0 

5.3 

Nonblack  males 

18.5 

20.9 

18.5 

22.0 

Black  females 

.2 

.2 

1.4 

1.4 

Nonblack  females 

.5 

.5 

1.8 

1.9 

Over  37  years  old 

20.4 

21.7 

10.4 

22.1 

Black  males 

1.8 

1.3 

3.3 

3.0 

Nonblack  males 

18.3 

20.2 

15.9 

18.2 

Black  females 

.1 

.04 

.4 

.2 

Nonblack  females 

.2 

.2 

.8 

.7 

NOTE:  Some  of  the  subtotals  for  particular  groups  may  not  add 

to  those  reported  in  Table  12,  owing  to  missing  data  and 
rounding. 
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Table  15 

POSTSTRATIFICATION  WEIGHTS  FOR  FORMS  1  AND  2  COMBINED 
ENLISTED  PERSONNEL  FILE,  BY  COMPONENT 


Cell 

Army  National 
Guard 

Army 

Reserve 

17  to  20  years  old 

Black  males 

1.12 

1.93 

Nonblack  males 

1.13 

1.44 

Black  females 

1.00 

1.50 

Nonblack  females 

1.00 

2.40 

21  to  24  years  old 

Black  males 

1.27 

1.61 

Nonblack  males 

1.04 

1.05 

Black  females 

1.67 

.80 

Nonblack  females 

.83 

.96 

25  to  29  years  old 

Black  males 

1 . 19 

1.12 

Nonblack  males 

.95 

.81 

Black  females 

1.33 

.89 

Nonblack  females 

1.00 

1.04 

30  to  36  years  old 

Black  males 

1.10 

1.13 

Nonblack  males 

.89 

.84 

Black  females 

1.00 

1.00 

Nonblack  females 

1.00 

.95 

Over  37  years  old 

Black  males 

1.31 

1.13 

Nonblack  males 

.91 

.87 

Black  females 

2.50 

1.50 

Nonblack  females 

1.00 

1.14 

NOTE:  Weights  are  derived  by  dividing  expected 
frequency  in  a  cell  by  observed  frequency.  See  Table 
11. 


IV.  APPLICATIONS  OF  THE  WEIGHTED  DATA 


Data  from  the  1979  Reserve  Force  Studies  Surveys  can  be  used  to 
study  the  enlistment  and  reenlistment  decisions  and  the  attrition  of 
Army  Reservists  and  Army  National  Guardsmen.  These  data  may  be  used, 
for  example,  to  expand  a  1978  Rand  study  of  reenlistment  decisions  of 
reservists,  which  focused  on  decisions  made  by  reservists  who  had  served 
from  3  to  8  years  with  no  prior  service.1 

The  1978  study  led  to  a  statistical  model  based  on  survey  data 
collected  at  the  point  of  the  reenlistment  decision.2  The  model  was 
based  on  actual  reenlistment  behavior.  The  work  showed  important 
determinants  of  reenlistment  to  be  the  demographic  composition  of  the 
reenlisting  cohort,  certain  characteristics  of  the  civilian  job, 
civilian  wage  rates,  reserve  pay  levels,  military  rank,  type  of  military 
job,  and  motivation  at  entrance.  Data  from  the  1979  Reserve  Force 
Studies  Surveys  can  be  used  to  extend  this  work  to  personnel  with  prior 
service  and  to  all  personnel  with  between  3  and  20  years  of  service. 

The  extension  of  the  1979  surveys  to  personnel  with  prior  service 
will  be  important  because  they  constitute  approximately  one-half  of 
reserve  accessions  and  their  reenlistment  behavior  is  known  to  be 
characterized  by  one-year  extensions  and  breaks  in  service.  Models  of 
both  prior-  and  nonprior-service  personnel  will  allow  improved 
determination  of  the  optimal  accession  mix  for  the  two  groups,  as 
determined  by  the  long-term  costs  and  reenlistment  behavior  of  each 
group . 

An  important  extension  allowed  by  the  new  data  is  the  modeling  of 
multiyear  reenlistment  behavior  for  both  groups.  The  1979  surveys 
collected  20  years  of  historical  data  on  participation  in  either  active 
or  reserve  components.  These  data  can  be  used  in  two  ways.  First,  one 

lSee  David  W.  Grissmer,  Zahava  D.  Doering,  Jane  Sacha  ,  he  Design, 
Administration ,  and  Evaluation  of  the  1978  Selected  Reserve  Reenlistment 
Bonus  Test,  R-2865-MRAL,  July  1982. 

2See  Burke  K.  Burright,  David  W.  Grissmer,  Zahava  D.  Doering,  A 
Model  of  Reenlistment  Decisions  of  Army  National  Guardsmen ,  R-2866-MRAL, 
October  1982. 


might  hypothesize  that  previous  breaks  in  reserve  service  would  be 
important  predictors  of  future  service.  Thus,  the  information  on  past 
history  can  be  included  in  the  reenlistment  model. 

The  information  can  also  be  used  to  explore  the  delay  between 
separation  from  the  active  force  and  enlistment  in  the  reserve  forces. 
Prior  service  reservists  often  do  not  join  the  reserve  unit  until 
several  years  after  active  force  separation.  Closing  this  gap  can  be 
important  to  reserve  strength  levels,  but  little  is  known  about  the 
length  of  or  reasons  for  the  delay. 

Knowledge  of  this  kind  will  be  particularly  important  in 
determining  how  long  the  large  Vietnam  era  pool  of  prior  service 
personnel  will  be  available  to  the  reserves.  It  will  also  suggest  how 
to  implement  policies  to  attract  a  larger  share  when  the  decline  in  the 
pool  occurs. 

The  extension  of  reenlistment  modeling  to  include  more  senior 
reservists  will  also  allow  the  evaluation  of  the  importance  of  reserve 
retirement  benefits  in  reenlistment  decisions.  The  effects  of  reducing 
benefits  may  be  predicted  from  such  models. 

Data  from  the  reserve  force  surveys  also  contain  a  richer  set  of 
potential  determinants  of  reenlistment  behavior.  In  particular,  many 
variables  describing  the  unit  and  community  environment  are  available 
for  analysis,  including  training  and  equipment  characteristics,  unit 
morale  and  manning  levels,  and  workloads  and  unit  resources. 

One  important  methodological  advance  that  can  be  addressed  with  the 
data  concerns  the  use  of  intentions  versus  behavior  in  modeling 
reenlistment  behavior.  The  survey  collected  intentions  to  reenlist 
based  on  a  ten-point  probability  scale.  By  matching  the  survey  data 
with  personnel  files  through  social  security  numbers,  the  actual 
reenlistment  behavior  can  be  ascertained  and  compared  with  the 
intentions  data. 

Work  with  active  force  data  has  established  that  intention  data  are 
quite  reliable  and  provide  estimates  similar  to  coefficients  of 
reenlistment  models.3  One  would  probably  hypothesize  a  weaker 

3See  Winston  K.  Chow,  J.  Michael  Polich,  Models  of  the  First-Term 
Reenlistment  Decision ,  R-2468-MRAL,  September  1980. 
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connection  between  the  intentions  and  behavior  of  reservists,  owing  to 
the  weaker  attachment  (full  versus  part-time)  to  the  military  job  and 
greater  uncertainty  in  civilian  job  demand  and  family  attitudes. 
Therefore,  a  finding  for  reservists  similar  to  that  for  active  forces 
would  allow  improved  forecasting  of  reserve  reenlistment  behavior  as 
well  as  easier  development  of  reenlistment  models. 

The  1979  survey  data  will  also  allow  a  more  complete  analysis  of 
reserve  attrition  behavior.  Analysis  to  date  has  been  restricted  to 
statistics  collected  from  reserve  personnel  cohort  files.  While  these 
files  can  provide  excellent  data  on  the  longitudinal  history  of 
individual  reservists,  they  contain  a  limited  set  of  variables  that  can 
be  used  to  model  attrition.  Only  demographic  variables,  Armed  Forces 
qualification  test  data,  and  military  rank  and  occupation  are  available. 

The  new  data  provide  information  on  the  civilian  labor  force 
status,  characteristics  of  the  civilian  job,  civilian  wage  levels, 
family  and  marital  status,  and  employer  attitudes  towards  the  reserve. 

In  addition,  they  offer  many  variables  describing  the  unit  environment, 
including  training  and  equipment  characteristics,  morale,  and  manning 
levels.  By  matching  survey  and  personnel  files  to  determine  which  of 
the  survey  participants  have  left,  fairly  comprehensive  attrition  models 
can  be  developed. 

The  unit  survey  contains  descriptive  data  on  a  recent  nonprior- 
service  attrition  decision.  These  data,  provided  by  the  unit  commander 
or  technician,  allow  an  evaluation  of  the  individual  who  left  and  the 
reason  for  his  leaving.  We  do  not  now  know  how  much  attrition  is  the 
normal  winnowing  of  undesirable  personnel  initiated  by  unit  commanders 
and  how  much  is  the  loss  of  desirable  personnel. 

In  addition  to  analyzing  individual  behavior,  the  survey  data  can 
be  used  to  explore  unit  locations.  It  has  also  been  suggested  that 
reserve  units  and  requirements  are  not  geographically  distributed  to 
take  advantage  of  population  distributions  and  the  different 
propensities  to  enlist  among  different  segments  of  the  population.  This 
suggestion  had  led  to  proposals  for  a  geographic  realignment  of  units 
and/or  requirements.  Geographic  realignment  would  lead  either  to 
increased  manning  at  the  same  overall  costs  or  to  the  same  manning  at 
reduced  costs. 
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Current  proposals  to  expand  the  reserve  forces  would  also  require 
decisions  regarding  new  unit  locations.  One  important  consideration  in 
this  decision  will  be  the  likelihood  of  acquiring  needed  manning  levels 
from  the  community.  These  survey  data  can  be  used  to  model  the 
relationship  between  unit  manning  levels  and  community  characteristics 
while  holding  constant  certain  unit  characteristics.  Census  data  are 
being  prepared  to  merge  with  unit  data  so  that  a  more  complete  picture 
of  the  community  can  be  obtained. 

Finally,  the  Unit  Commander  surveys  can  be  used  to  characterize 
reserve  unit  commanders  and  obtain  their  assessments  of  reserve  problem 
areas  for  different  kinds  of  reserve  units.  Such  assessments  were 
obtained  for  personnel,  equipment,  and  training  resources.  Many  believe 
that  the  experience,  management  skills,  and  dedication  of  unit 
commanders  explain  much  of  the  variance  in  unit  readiness  and  manning 
levels.  Data  from  this  instrument  can  be  combined  with  unit  date,  to 
test  the  hypothesis  that  various  characteristics  of  the  unit  commander 
are  important  to  these  variables. 
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